Robotic systems that occupy a physical environment, such as control systems for robotic devices, may maintain a map of the environment in order to facilitate robotic and/or human action within the environment. In some cases, a robotic system may acquire visual information about the environment and use that information to construct the map, which may include locations of obstacles, robotic devices, and other objects, for instance. Robotic devices may then navigate through and perform actions in the environment using the constructed map.